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(g) Recording and retrieval of information relevant to the activities of a user. 

@ A method for the recoiding and retrieval of 
information relevant to the activities of the user. 
The invention may be employed to provide a 
memory prosthesis in the fonn of a highly port- 
atHe device (30) (e.g. a device worn like a watch) 
which records all kinds of information about the 
user's life, and provides a simple and rapid 
means of access to that information. The device 
receives and stores information from external 
sources ('objects') via, for example, a cellular 
communications network, or from signal 
generators identifying specifk; machines, 
places, or people carrying other ones of the 
same type of device. Events comprising en- 
counters between 'objects' are recorded auto- 
matically, being time-stamped and stored 
chronoiogicaJly, and a rapid search method for 
such a database is disclosed. The data can 
either be stored tn the memory of the device 
itsetf, or, for better protection against failure, in 
a central data repository of the system, with a 
faa-safe backup. 
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This invention relates to a syst m and method for the recording and retrieval of information relevant to the 
activities of the user. Th inv nti n is particularly suitable for impi mentation in the form of a memory aid device 
which Is portable, and preferably wearable by the user. 

IVIemory problems are widespread but little has been done to exploit information technology to help. People 

5 have many procedures designed to compensate lor thelrfailible memory, for example, writing lists, organising 
things in files, putting notices on notice boards and setting alarms. Even in a well-organised setting, things 
might be mislaid or forgotten because they were not seen to be important at the time they were encountered. 
These failures of memory have a cost, both in the time spent recovering (such as looking for mislaid docu- 
ments) and in the time spent organising things in the first place so that problems do not arise. 

10 One current form of memory aid is the electronic personal organiser, which is similar in form to a hand- 

held calculator. Although such a device is capable of holding large amounts of infomnation. each item must be 
entered, usually manually, so the user must both input the information and remember to do so. 

Various other systems have recently been proposed to help the user recall past events as well as to provide 
reminders of future intentions. For example. In EP-A-G.495,622 there is described an activity-based infbmria- 

15 tion retrieval system which automatically captures and records certain aspects of the context in which a re- 
corded event took place. The system relies on a variety of apparatuses for monitoring, recording and time- 
stamping various key aspects of human activity, from which a readable list of episodes, or 'activity diary' can 
be generated automatically. In one example, the system uses encoded identifiers, each intended to be carried 
by people worthing in the environment being monitored. One particularly convenient form of identifier is the 

20 "active badge" offered by Olivetti. This identifier takes the form of miniaturised circuitry in a housing able to 
be pinned or dipped to the clothing of the person to whom the identifier is issued. The circuitry is designed to 
erhit pulse width modulated infra-red coded signalsfor a tenth of a second every 12 to 15 seconds. The signals, 
by their nature, have a range of about sbc metres, and will not travel through walls, which makes them very 
useful for keeping track of people moving between rooms In a normal working environment In each of the 

25 rooms or corridors of interest would be positioned one or more sensors responsive to the infra-red signals. 
The sensors would be connected to a master statfon processor, whteh would have the task of polling the sensors 
for identification 'sightings'; processing the data (which would include decoding the signals to determine which 
person was within range of a specified sensor at the time of polling), and presenting the processed data in a 
visual format in the form of a diary. This diary, which conrelates quite closely with human recollections of past 

30 episodes, can be used simply to stimulate further recall, or to identify an episode and its time of occurrence, 
or to show a set of episodes from' different sets. The time may be used to locate the corresponding items in 
any other set of time-stamped data, such as a sequence of video frames or an audio record of a meeting. 

A related system was disclosed in an article in The Sunday Times, 11 February 1 9g0, entitled "Computers 
acquire social skills*, by Jane Bird. This mentioned the use of identity buttons to enable the computer to know 

35 who was where in the building at all times. Associated with the system is a network of microphones and/or 
video cameras capable of recording cx^nVersations. It gave the example of a copier in a room monitored by the 
computer. Upon the copier's developing a fault, a frustrated woUld-t>e user tries to recall what a colleague told 
him about how to cure the fault The computer is Interrogated to find the last time that the two people concerned 
were at the copier; Once this particular bit of information has been recovered, it is used to access the tlme- 

40 stamped audio records of what was said in the room containing the copier at that point of time In this way, the 
enquirer has the opportunity of hearihg' again the cohversatk>n between him and hte colleague. 

The aforementioned systems have several' practical limitations. These include the need on the part of the 
user to devotee lot of effort tbidentifying-a particular tinne and place from the law data, and the lack of precise 
episode definition from such data. Also the systems capture just the location of users, and not their activity or 

45 other significant episodes. These drawbacks would make the systems of relatwely little practical value. 

There is still, therefore, a need for an improved memory akJ system which requires minimum input from 
a user, and yet whtoh is easier to use for retrieving infonmation than to remember or find whatever it Is that has 
been forgotten or mislaid. It is an object of the present invention to meet this need. 

According to the present invention, there is provided an information recording system comprising: 

50 a plurality of objects, each object representing a human user or other entity identifiable by the system 

and each object comprising an agent, means for communicating between the agent and the agents of other 
ones of the objects, and, in communication with the agent, at least one of a database, a sensor, and a user 
interface; 

the system including at I ast one databas ,atl ast one sensor, and at I astoneus r interface; 
55 at least part of th ag nt of at least one of the objects representing a human user being portable by th 

us r, providing interaction with the user, and Identifying to the system the location of the user; 

the infonhation comprising, for a given object, a chronological sequ nee f time-stamped facts in its 
database, each fad comprising an attribut and a value of that attribute for a given object at th tim indicated 
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by the time Stamp; and 

the.fects being automatically recorded in the database oh the occurrence of physical or logical colloca- 
tions of two or more of the objects. 

According to another aspect of the invention, there is provided a method of searching a database in which 

5 data is organised as a set of threads, each thread comprising a chronological sequence of time-stamped ^cts 
all of which relate to a single object and at least-some of which also represent descriptors of other objects, 
each object representing a person or an identifiable entity, and each of the facts comprising an attribute and 
a value of that attribute for a given object, the method indudihg displaying.the sequence of facts of a selected 
thread, or the sequence of events defined by the coincidence of twb ormore threads, and moving selectively 

10 forward or backward In time through the displayed sequence of. facts or events, and the method including ena- 
bling the selection, for subsequent display, of any one of the threads of a displayed event. 

The invention also provides an apparatus for recording facts relevant to the activities of the user of the 
apparatus, including a database, a user interface, and means for receh^ing signals from externsd sources, the 
signals representing facts related to the isources; each of the facts being time-stamped and comprising an at- . 

IS tribute and a value of that attribute for a given source at the time indicated by the time stamp, and the facts 
being automatically recorded in a chronological sequence ln;the database on the occurrence of physical or 
logical collocations of the apparatus and one or more of the sources; and the apparatus being operable through 
the user interface to enable the user to retrieve from the.datat>ase a sequence of occurrences of a selected 
fact or pattern of fiacts. 

20 Preferably, the apparatus of the preceding paragraph include means to enable nianual entry of the facts 

or other information Into the database, and Is wearable by the user. 

The invention may be employed to provide a memory prosthesis in the form of a highly portable device 
(e.g. a device worn like a watch) which records all kinds: of information about the user's life, and provides a 
simple and rapid.means of access to that informatton^The device receives and.sfores information f rom external 
25 sources ('objects') via, for example, a cellular communications network, or from signal generators identifying 
specific machines, places, or people carrying other ones of the same type of device^ The data can either tie 
stored in the memory of the device itself, or, for better protection against failure, in a central data repository 
of the system, with a fail-safe backup. ] .» 

Data Is gathered continuously with no attempt being made to categorise or filter it at the time of poUection:. 
30 This means that all the data is retained, so that items that might have been. categorised as insignificant at the 
time of collection can later be retrieved. As a result there will be. large quantities of d9fa for the user to search. 
For the memory prosthesis to be usable, it must provide a mechanism for enabling the user to constrain the 
search and rapidly find the relevant information. This mechanism is supported by the data model described 
below. ■ V «\. •■ t • \- .. • 

35 The data consists of series of events which happen to each object in the system,. Each object has its own 

"thread", a linear organisation in time of the events occurring as the. object encounters other objects. Access 
to the data is achieved by displaying sequentially; running either forwards or backwards in time, the series of 
events along a selected thread. Rapid searching is made possible by. specif yhig events formed by the coinci- 
dence of several threads, or by changing from one thread to another at a event where the two threads intersect , 
40 The memory prosthesis records all kinds of information about the user's life,-and.by providing a simple 

and rapid means of access to this information It can help the- user, to recall things that might otherwise be for- 
gotten. All information in certain specified categories is gathered, -without filtering, so that even things that 
might not have appeared Important at the time of recording can later be recovered: , 

A system and method in accordance with the invention will now be described, by way of example, with 
45 reference to the accompanying drawings, in which:- 

Fig. 1 Is a diagrammatic representation of the elements which comprise an -object"; as defined herein; 
Fig. 2 is a pictorial representatton of an example ofa system in accordance with the invention; . 
Fig. 3 IS a diagrammatic representation of the threads of several objects, . . 
Fig 4 illustrates a set of 'threads' to show how a user can navigate the information in a database; 
50 Fig 5 shows an example of a user interface; and 

Fig. 6 is a flow chart shovying how data is processed by one of the 'objects' used \n the system of the in- 
vention. 

Referring to Fig. 1 , the basis of the information recording system comprises at least two 'objects' 1 0 of the 
kind shown, which together def in a domain. The domain may cov r, for .example, a room, a . building, all the 
55 buildingswithtnagroupof buildings, or a larger ar a.Anobj ctlOr presents a person or another identifiable 
entity such as a location, an apparatus or machine, a vehicl . a piece of mall, som t xt, .a number, a time. 
Th system thus c mprises a set of obj cts 1 0, within the domain, for which a hist ry can be generated auto- 
matically. Each object 10 conforms to the archit cture shown in Figur 1. All obj cts 10 have a database 12 
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in which they store descripti ns of the activities in which they were involved. A whiteboard might store the 
names of people using it, derived for example from active badge identifiers, and snapshots of its contents at 
periodic intervals. Associated with each object is an agent 14 which has three main functions: it manages the 
datat>ase 12; it exchanges information with other objects over a communications channel 16; it communicates 

5 with human users over a user interface 18 (Ul) to answer queries using the database; f inally, it collects data 
from Its own sensors 20. While all objects 1 0 have an agent 1 4, a database 12, and a means 1 6 to cornnminicate 
with other objects, they miay not possess a Ul if they do not provide an interface for querying ttieir database. 
A room-conversation detection object is an example. Some objects 10 may not contain sensors if they obtain 
all their information from remote sources over the communications channel. 

10 Referring now to Figure 2, there is shown a system containing a number of objects which form a domain 

including a personal, portable, computer 30 which may be described as a 'tab'. This tab includes an agent, a 
database, a Ul which includes a screen 32 and control buttons 34, and communication means which are pre- 
ferably wireless, such as radio or infra-red. Jane, the person whom the tab represents, carries or wears the 
tab as she moves around the domain and cam'es out various tasks. The tab here is the object which represents 

15 Jane. When she encounters (e.g. approaches, enters, or uses) one of the other objects, a signal between 
Jane's tab and the other object generates a time-stamped record in the database of Jane's tab (and possibly 
also in the database of the other object). The record includes the time of the encounter (the time-stamp), and 
identifies the primary object (Jane), an attribute of that object (e.g. location - -af )^ and a value of that attribute 
(Jane's Office). 

20 By way of example, let us assume that Jane enters her office one day at 10:1 5. Here, the record in her database 
wilt show "10:15 (time-stamp) - Jane (object) - at (attribute) - Jane's Office (value of attribute)". The screen 
on her tab can use either woras or Icons to display this record, or a mixture of the two. if Steve enters Jane's 
office at 10:47, a record is generated to show that Steve and Jane are collocated in Jane's office. When Jane 
uses the telephone at 10:53 to oall Bob, the objec^ representing her has the attribute 'using' and the value tei- 

25 ephone'^ The record wilt show Jane, Steve, and telephone being used by Jane (to call Bob), in Jane's off ice. 
Jane, makes a note during the telephone call. This note may also be recorded in Jane's database, either auto- 
matically if she made it, say, on a workstation which is anotherof the objects in the system, or manually using 
her tab. Jane then finishes the telephone call, and Steve leaves her off ^ . 
If, many months later, and using the example just described, Jane wishes to recall the note she made during 

30 her telephone conversation, she can ask; her teb to show her a list of all her telephone calls. This might be un- 
manageably long, so she might remember that she made the note while on the telephone at a time when Steve 
was in her office. Thus she could ask her tab to list for her all those occasions when she was using the telephone 
while Steve was In her office. This would considerably limit the search (assuming Steve was an infrequent 
visitor), and the tab screen would show just those j(few) occasions which would include the one that Jane was 

35 seeking. She could then identify the note, for example by its time-stanr>p, and display it on her teb. 

The objects shown in Figure 2 have different configurations and requirements. It is most likely that all the 
human users will need a datebase and a Ul, biit rhuch less likely that the other objects, like off ices or machines, 
will need them. However, it is sometimes useful for machines to have both, so that records of who used which 
machine at what time can be kept and later reviewed. 

40 The following definitions and explanatlpns are provided to assist in understending the invention. 

DATAlMODEL 

information is held in the datebase as a sequence of time-stemped facts. . 
45 Each fact consists of an object, an attribute and a value. 

A fact represents the knowledge that the attribute of the object is the value. 
The time-stamp indicates the time at which this fact became known. 
The sequence is held in chronological order In a relational database. 

An object represents either a person or any other entity which Is identifiable by the system, such as a lo- 
50 cation, an apparatus or machine, a yehide, a piece of mail, some text, a number, or a time. 

An attribute indicates one of the possible properties that some object may possess, such as hiair colour, 
whereabouts, or size. 

A value represents the particular value of an attribute, such as black for hair colour, the kitchen for where- 
abouts, or large 'for siz . (See Tat>le 1 below). 

55 
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Time 


Object 


Attribute ' 


Value 


11:05:26 Mon 21st June .1993 


Steve. 


at.. 


Kitchen 


11:37:13.Mon!2ist June 1993 


Jane , 


at / 


Jane's Office 


11:39:22 Mon 21st June 1993 


Jane 


at"..'' 


Kitchen 


11 :42:35 Mon 21st June 1993 ' 


Jane '., 


at ; 


Jane's Office 


12:01:04 Mon 21st June 1993 . 


Jane 


availability 


Busy 


12:01:09 Mon 21st Jun^ 1993 


Kitchen 


temperature 


22 Celcius 


Table 1, Typical time-damped facts in a database. 



The episode which Table 1 represents can be described as follows. Steve enters the kitchen at 1 1 :05. Jane 
enters her office at 11:37, and then goes into the kitchen at il:3g>. She returns to herofftoe at 11:42, and is 
recorded as being 'busy* (e.g. with a telephone call) at 1 2:01 . Since no change lias t>een noted in Steve's 'af at- 
tribute, it can be assumed that he remained in the kitchen throughout this period of time. It is possible to infer 
from this that while Jane was in the kitchen. Steve and Jane were collocated at 11 :39. This inferred fact is'shown 
in Table 2. 
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INFERENCE 

An agent may apply inference rulestothe infonmatioh thus held, in order to establish otherfacts not directly 
recorded, such as when two people were in the saine room, or the duration of a meeting. 

Such inferred facts may be determined by the agent upon request, or determined in advanbe and held in 
the database on the same footing as ordinary fadts. 

inferred facts may be used during inference, to produce further inferred facts. (See Table 2 below). ' 



Time 


Object : 


Attribute i 


Value 


11:05:26 Mon 2l8t June 1993 


Steve ^ 


at-^--..:: 


Kitchen 


11:37:13 Mon 21st June 1993 , 


Jahie 


■'at ' 


Jane's Office 


11 :39:22 Mon 21st Jiine 1 993 


Jane - 


■^at - - ■ 


Kitchen 


11:39:22 Mon 21st June 1993 


Steve ; 


co-locked ^ , 


Jane 


11:42:35 Mon 21st June 1993 


Jane 




Jane's Office 


12:01:04 Mon 21st June 1993 


Jane * * 


availability'' 


Busy ' 


12:01:09 Mon 21st June 1993 


Kitchen 


temperature 


22 Celcius 


Table, 2, An exp'a inferred fact intmduced, in italics . . 



PATTERNS 



SO 



ss 



When a request for Information is made to the agent, a pattern may be specified. . . 
A pattern consists of either, a conjunction of patterns: a disjunction of patterns; a negation of a pattern; 
or an object 

Such a pattern filters the information yielded by the agent to include only those facts which conform to 
the pattern. 

A fact conforms to a conjunction of patt rns if it conforms t each of th conjoined patterns indivkjually. 
A fact conforms to a disjunction of patterns If it conforms to any one or more of th disjoined patterns. 
A fact conforms to a negated pattern if it does not conform to the patt rn which is negated. 
A fact conforms to an object pattern, if the fact mentions the object of the pattern. 
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SEARCH 

In response to infpnnation request, the agent searches for particular facts by considering the whole of the 
sequence of facts and making those iriferences possit>le. to find facts conforming to the supplied pattern. 
. In principle, the agent checks every fact, simple or inferred, for conformity with the pattern. 

In practice, the agent need only consider those facts pertinent to the pattern that is, the contents of the 
pattern may t>e used to inform and constrain the possibly huge search under^ken by the agent to reasonable 
dimensk>ns. 

For example, a pattern which specif ied that a particular person, place arid day of the week must be men- 
tioned, allows the agent to consider only those facts and inferred facts pertinent to that person, place and day, 
thus greatly reducing the scope of the required database search and the number of facts which must be tested 
for conformity to the pattern. * . * 

THREADS 

IS 

If a pattern contains at least one object, then the only facts that the agent will yield are those pertinent to 
that object Such a chronological sequence of fects may k>e considered a tHography of the specified object,, or 
thread. 

Asimple user-interface paradigm would be to keep at least one object in the pattern at all times, and display 

20 a thread for that object Siich a thread will mention other objects, such as locations or other people. One of 
these other objects nrtay be selected by the user as the new focus, and the pattern used from then on would 
contain the new object Thus a switch In perspective occurs, to the ttiread of the newly selected object - the 
biography displayed Is now that of the new object 

The world is thus seen as consisting of a variety of objects, each one of which automatically generates 

25 an autobiography recording events in its "life". These objects could in prindple be anything, t>ut typical exam- 
ples are people (at least, computers attached to them), offices, telephones, notebooks and workstatk>ns (see 
Figure 2). The autobiography for each object consists of a series of events that happened to that object, each 
event being an encounter of this object with another. These events are organised linearly in time as a "thread". 
For example, as Steve walks around his thread 40 shows hirn in his office 42, then the stairwell 44, and then 

30 the common room 46 (see Figure 3). Conversely, his office thread 42 shows him 40 enter, stay for a while and 
then leave. The stairwell thread 44 shows him 40 enter and leave and finally the common room thread 46 shows 
him arrive. A further thread 48 belonging to Jane shows her visiting Steve 40 whilst he is in his office 42. This 
visit also appears on Steve's own thread 40 and that of his office 42. During this period Steve interacts several 
times with his electronic notebook 50. Each interaction appears on the thread of the notetxook 50 and also on 

35 Steve's thread 40 and the thread for the room in which he is usirig the nbtet>ook. Whilst he is in the common 
room 46, Steve uses the copier 52. This ihteractkm also appears on his thread 40, and those of the conmon 
room 46 and copier 52* 

Threads that run together indicate coZ/ocaton of the objects that own the threads. The collocation could 
also be taken to mean some other sort of relationship such as "interacted with" (e.g. via telephone or worksta- 
40 tion), thus representing a log ical:Collo<^t ion rather than a physical one.. The significant points recorded on a 
thread are the points where threads join together or split apart, Indk^tlng a transition from one state of affairs 
to another. 

Users can retrieve tnf(»matk>n about events in ther life by finding the relevant points in the threads struc- 
ture. Starting from any point they can navigate around this structure by moving forward or back between 

45 events along a particular thread. For example, if Steve had mislaid some papers given to him by Jane during 
her visit to his office he could prompt his memoi'y by looking on his thread and finding Jane's departure from 
his office. From here he could rnove along the thread to see the sut>sequent events: him leaving his office, 
entering and leaving the stairwell and entering the dbmmbn room. He thert sees that the next event on his thread 
records him using the copier and this reminds him that he left the papers on the tat>le beskle the copier. 

50 The user can also switch onto another thread at a point where the current thread intersects it Foir example, 

if Steve found someone's originals In the copier when he came to use it he could switch from his thread to that 
of the copier. By following back down the copier thread he could find the previous person who used the copier 
and hence return the oiiginals. 

These examples have beeri V ry simple, but in reality there will t>e a large number of v ntsihanythr ad 

55 that is t be navigated. To simplify navigation th user can spedfy which particular vents are of Interest by 
giving a pattern to be matched. Such patterns can specify any combination of location, people, time or oth r 
objects. For example, if Stev specifies "iri my office with my notebook with Jane" he will s e only those c- 
casions that match this pattern i.e. those ccasions when he was in his off ice with Jan aindmad som not s. 
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The actual not s, which may have been recorded in the databas by the user, ar linked to these events, thus 
allowing him to retriev th m. 

By way of exannple of a liser 'navigating' the database using the threads model, reference is made to Figure 
4 in conjunction with Table 3. Figure 4 shows a sequence of sieven events, labelled in chronological order of 
their occunrence from 1 tio 7. Eabh event comprises the conju notion (Intersection) of two or nrrore of threads A. 
B, and C. Event 1 shows a coihddence of threads A, B and C, event 2 involves only threads A and B, event 3 
involves thr^ds A and C, and so on. In searching the part of the database invoking these events, the user 
might 'browse' in the manner represented by way of example in Table 3. In the first step, he starts at event 1 , 
moves forward (in time) along thread A and arrives at event 2. In step 2 he nrbves backwards £dong thread A 
and returns to event 1. In step 3 he changes to thread B. In step 4 he moves forward along thread B to arrive, 
on thread iB. at event 2. In step 5 he moves forward along thread B to arrive at event 4. In step 6 he changes 
to thread C. In step 7 he moves forward along thread C to arrive at event 6. 

Instead of navigating along single threads, as just described, the user may select patterns of threads, the 
simplest pattern being a conjunction of two or more threads. I step 1 0, search is requested by the conjunction 
of threads A, B and C. Starting at event 1 , which is the first such conjunction, the user moves forward to event 
7, which is the next such conjunction. In step 11, a change is rnade to a pattern consisting of the conjunction 
of threads A and B. In step 12, the user moves backwards through such patterns, the next occunrence of an 
A,B con^nction (going backwards) being event 5. 



Step No. 


Event 


Search Direction 


Thread . 


. Result 


1 


1 


Forward 


A 


2 (on A) 


2 


2 


Backward 


A 


1 (on A) 


3 


1 


Change to B 


A 


1 (onB) 


.4 


.1 . 


Forward 


B 


2(onB) 


5 


2, 


Forward 


B 


4 (on B) 


6 


. '4. 


Change to C 




4(dnC) 




4 


Forward 


■■■ ..^;vC:. . 


: 6 (on C) : 












10 


1 / 


. Forward . 4^ . 


^;'/v:B;a.- 


~7 (onA, B, C) 


11 


7 


Change to A,B 


A, B. C 


7(onA,B) 


12 




Backward 




^ 5(pnA.B) 


Table 3, Exampi0 of navigating datai in a iHr^^ 



USER INTERFACE 



45 



SO 



55 



Figure 5 gives one simple example of, how the user interface rnight be organised for the memory prosthesis. 
The physical buttons on the side of the device can be used for nrioying forward or back through the cunrent 
thread. Touching one of the on-screen buttons associated with intersecting threads switches to that thread. 

The device shown in Figure 5, in the fomri of a 'tab*, can be large enough for words to be used on its screen. 
However, if a smaller device, such as one in the same form as a watch, will need to have its Ihfoirmation dis- 
played In a highly codified fashion. I this case, each episode or event listed occupies a single line containing 
a sequence of icons. The first item in each line represents the start tinrie of the episode. It is followed by an 
activity indicator, representing the type of activity, such as a meeting, a time spent doing paperwork, or per- 
haps a vacation. The rest of th line contains a sequence of icons, each of which represents an object which 
the user encountered in the displayed episode. The top line of such a display contains information which applies 
to the whole of th history.. It includ s an icon representirig the owner of th history display d. It also includes 
the k:ons representing th filter pattern. The rest of the line contains the list of it n^s thiat ar conunon to all 
the episodes listed below it. This avokis repeating icons in each episode descrii>tion, thereby economising on 
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sere n space. 

in reality any such user interface would be generally used as part of som other application, in a similar 
way to the Finder dialogues on the Macintosh: It would provide a standardised way for the user to locate in- 
fomnation using biographical infomDation. The particular application would dictate what happened to the infor- 
mation retrieved. 

Referring now to Figure 6, there is shown a flow diagram illustrating the interactions of the main constit- 
uents of one of the objects which form part of the system of the invention. 



Claims 

1. information recording system comprising: 

a plurality of objects, each object representing a human user or other entity identifiable by the sys- 
tem and each object comprising an agent, means for communicating between the agent and the agents 
of other ones of the objects, and, in communication with the agent, at least one of a database, a sensor, 
and a user interface; 

the system including at least one database, at least one sensor, and at least one user internee; 

at least part of the agent of at least one of the objects representing a human user being portable 
by the. user, providing interaction with the user, and identifying to the system the location of the user 

the information comprising, for a given object, a chronological sequence of time-stamped facts in 
its database, each fact comprising an attribute and a value of that attribute for a given object at the time 
indicated by the time stamp; and 

the facts being automatically recorded in the database on the occurrence of physical or logical col- 
locations of two or more of the objects. - 

2. The system of daim^l including means to enable manual entry of the facts or other information into the 
database. 

3. The system of claim 1 or claim 2 further including means for automatically recording facts iri the database 
at predetermined times. . 

4. The isystem of any one of claims 1 to 3 if urther including rrieans for automatically recording facts in the 
database on the occurrence of predetermined values of an attri^ 

5. A niiethod of searcHing a diatabase in which daita is brgarii^ed as a set of threads, each thread comprising 
a chronological sequence of time-stamp^ facts al! of which relate to at single object aiid at least some 
of which also represent descriptors of other objects, each object representing a person or an identifiable 
entity, and each of the facts comprising an attribute and a value of that attribute for a given object, the 
method including displaying the sequence effects of a selected thread, or the sequence of events defined 
by the coincidence of two or more threads, and moving selecth/ely forward or backward in time through 
the displayed sequence of facts or events, and the method including enabling the selection, for subse- 
quent display, of any one of the threads of a displayed event 

6. . An apparatus for recording facts relevant to the activities of the user of the apparatus, including a data- 

base, a user Interface, and means for receiving signals from external sources, the signals representing 
facts related to the sources; each of the facts being time-stamped and comprising an attribute and a value 
of that attribute for a given source at the time indicated by the time stamp, and the facts being automat- 
ically recorded in a chronological sequence in the database on the occurrence of physical or logical col- 
locations of the apparatus and one or more of the sources; and the apparatus being operable through 
the user interface to enable the user to retrieve from the database a sequence of occurrences of a selected 
fact or pattern of facts. 

7. The apparatus of claim 6 including means to enable manual entry of the facts or other Information into 
the database. 

8. The apparatus of daim 6 or daim 7 wherein the user interface nabl s the user to search by displaying 
sequentially the occurrences of a selected fact or pattern. 

9. The apparatus of any one of daims 6 to 8 wherein each sequ nee of occurrences of a fact comprises 
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a'thread'.wher in achofth facts is repres nt d by th coincidence oftwo or more fth threads, and 
wherein the user.can search the database by selecting one thread, or by selecting the coincid nee of a 
plurality of threads, or by moving fron) one thread to another. 

5 10. The apparatus of any one of daims 6 to 9 wherein the apparatus is wearable by the user. 

11. A system comprising the apparatus of any one of daims 6 to 10 and a plurality of the sources arranged 
to cooperate with the apparatus. 

12. The system of daim 11 induding a plurality of said apparatuses, at least some of the apparatuses com- 
prising at least some of said sources. 

1 3. The system of claim 11 or daim 12 wherein at least some of the sourbes comprise a cellular communica- 
tions network; enabling a device to deterrhine its position in the network. ' 

IS 14. The system of any one of daims 11 to 13 wherein the, or each, apparatus is in oommunteatton with a cen- 
tral data processing unit 

15. A system composing : 

a piuraMty of signal sources; each signal source providing signals that Identify the signal source; 
20 a first one of the signal sources being portable so that the first signal source can be located near a second 

one of the signal sources; the first signal source providing signals that Indicate location of the first signal 
source. 

signal receiving circuitry for receiving signals from the signal sources and for providing signal data 
indicating information from the signals; upon receiving a signal from the first signal source indicating that 
25 the first signal source is located near the second signal source, the signal receiving circuitry providing 

collocation signal data indicating that the first signal source is located near the second signal source;. . 

memory for storing data; and 
processing circuitry connected for receiving signal data from the signal receiving drcuitry and also con- 
• nected for accessing the memory; ' ^ • ; 

30 upon receiving signal data indicating a signal from one of the signal sources, the processing cir- 

cuitry using the signal data to obtain event data, the eyent data indicating the signal source and a time 
at which the signal data were received; ~! 

upon receiving the collocation signal data, the processing circuitry using the dollocation signal data 
to obtain collocation event data, the collocation event data indicating the f irst signal source, .the second 
35 . signal source, and the time at which the collocation signal data were received. 
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@ Recording and retrieval of information relevant to the activities of a user. 



(g) A method for the recording and retrieval of 
information relevant to the activities of the user. 
The invention may be employed to provide a 
memory prosthesis in the form of a highly pot- 
able device (30) (e.g. a device wom like a watch) 
which records all kinds of information atx>ut the 
user's life, and provides a simple and rapid 
means of access to that information. The device 
receives and stores Information from extemal 
sources ('objects') via, for example, a cellular 
communications network, or from signal 
generators . identifying specific machines, 
places, or people carrying other ones of the 
same type of device. Events comprising en- 
counters between 'objects* are recorded auto- 
matically, being time-stamped and stored 
chronologically, and a rapid search method for 
such a database is disclosed. The data can 
either be stored in the memory of the device 
itself, or, for better protection against failure, in 
a central data repository of the system, with a 
fail-safe backup. 
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